Coordinate your interviews with assertion-based interview plans

Hiring well is the most important thing for companies. Today I share an easy way to plan interviews that almost guarantees better results.

How to coordinate poorly

The most common (and terrible) way to organize interviews is to:

• Decide on a group of people who can interview the candidate.
• Come up with a list of areas you want them to dig in on.
• Have someone schedule the time slots
• Have each person give a recommendation to hire or reject the candidate.

This is problematic in a million ways:

• When I’ve been on panels like this, I’ve had candidates tell me every person asked them the same questions. So the candidate experience is pretty terrible.
• You’re not really guaranteeing you are going to learn what you need to with the candidate.
• You have no idea of what each interviewer is doing, so you have no way of knowing if their feedback is solid or not. You have no visibility into the quality of your interviews.

The main problem is that there is no coordination.

Coordinate your interviews with assertion-based interview plans

So let me share an interview format I use.

The basic process is:

1. Make a list of assertions for an ideal candidate.
2. Create questions to test your assertions.
3. Group and assign the questions.
4. Tweak the interview format continuously

Make a list of assertions for an ideal candidate

Make a list of assertions that should be true for the person to be right for this role. For example, for an engineering manager, you might come up with a list that starts like this:

1. Has improved the organization they were a part of
2. Able to attract and hire great talent.
3. Experienced with org design and scaling organizations.

Roughly order the list by what is most important for the role. You won’t be able to test everything, so focus on what you care about.

Create questions to test your assertions

For each assertion, list an interview question (or action, like a programming assignment) that will uncover evidence of how strong the candidate is in this area. Indent the questions.

1. Has improved the organization they were a part of\

Tell me about the largest improvement you’ve made in your organization. I’m interested in process changes, or direction changes, or other things that had an impact on the company.

If you can, add what a good answer might look like. For example:

1. Has improved the organization they were a part of\

Tell me about the largest improvement you’ve made in your organization. I’m interested in process changes, or direction changes, or other things that had an impact on the company.

A poor answer would be switching to Kanban, or introducing standups (yes these can improve things, but the impact is very local). A better answer would be something like changing the way leadership worked together, or how people communicated, if the impact were good. A perfect answer would relate the impact to the results on the company, and the results would have made a substantial difference for the company.

Do this for your top questions. If you’d like to see excellent examples of what good answers look like, this article from Medium engineering is quite good.

Group and assign the questions

Now we flesh out the interview plan and assign people to it.

First, group your assertions by category, like Managing Projects, Managing People, Managing process, and Technical decision-making. These are your interview sections.

Your list will now look something like:

Interview Section Category A (Interviewer i1, Interviewer i2)

Assertion 1

• Question to test assertion
  • What a good answer looks like.

Copy this this Google Doc and put your assertions and questions into the format, adding the people who will be responsible for each section.

(optional) Choose two people to do each interview. They’ll observe the interview better, and learn from each other, which increases your organizations ability to select candidates over time. One is okay if you can’t find a second person.

Work with each interviewer to improve the questions they will ask. Tell them what you came up with is a starting draft. Then, you act as editor, read through the interview and think about the interview experience, and whether questions duplicate each other. Make sure you have confidence asking these questions will generate the answers you’re looking for.

Before you start the interview, copy and paste the questions you’re asking to search for bias in the questions you’re asking. I often find I’m gender coding in a way that might exclude women — for example, in a recent interview I used two masculine-coded words: “active” and “force”. Usually removing the gender coding also makes it more clear.

Tweak the interview format continuously

After every interview, encourage everyone to ask themself these questions:

1. Did I get the information I needed to make a confident assessment of the candidate? If not, I should change my “assertions”.
2. Did I understand the results of the assertion by asking the questions I did? If not, I should tweak the questions I ask.
3. Did I hear a particularly good or poor answer to the questions? If so, I may want to tweak the “what does a good response look like” section.

Why does this format work?

1. It separates out what you want to learn from how you are going to achieve it. This makes it easier to iterate on the interview format after every interview.
2. It is explicit about what you’re looking for in a good answer. This reduces bias, because you will have thought of it beforehand. This helps with interviewees that may not be the best interviewee, but are giving answers that indicate they might actually be the right person for the job.

I hope you find this useful. Let me know what you think!

Image by Adabara Ibrahim from Pixabay